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Abstract 

In Constraint Programming (CP), a portfolio solver 
uses a variety of different solvers for solving a 
given Constraint Satisfaction/Optimization Prob¬ 
lem. In this paper we introduce sunny-cp2: the 
hrst parallel CP portfolio solver that enables a dy¬ 
namic, cooperative, and simultaneous execution of 
its solvers in a multicore setting. It incorporates 
state-of-the-art solvers, providing also a usable and 
conhgurable framework. Empirical results are very 
promising. sunny-cp2 can even outperform the 
performance of the oracle solver which always se¬ 
lects the best solver of the portfolio for a given 
problem. 


1 Introduction 


The Constraint Programming (CP) paradigm enables to ex¬ 
press complex relations in form of constraints to be satis- 
hed. CP allows to model and solve Constraint Satisfaction 
Problems (C SPs) as well as Co nstraint Optimization Prob¬ 
lems (COPs) [Rossi et al, 20061. Solving a CSP means hnd- 
ing a solution that satisfies all the constraints of the problem, 
while a COP is a generalized CSP where the goal is to hnd a 
solution that minimises or maximises an objective function. 

A fairly recent trend i n CP is solving a problem by means 
of portfolio approaches [Gomes and Selman, 20011, which 


can be seen as instances of the more general Algorithm Selec¬ 
tion problem [Rice, 19761. A portfolio solver is a particular 
solver that exploits a collection of m > 1 constituent solvers 
si,... ,Sm to get a globally better solver. When a new, un¬ 
seen problem p comes, the portfolio solver tries to predict the 
best constituent solver(s) ,..., Si^ (with 1 < A: < m) for 
solving p and then rans them. 

Unfortunately —despite their proven effectiveness— port¬ 
folio solvers are scarcely used outside the walls of solvers 
competitions, and often confined to SAT solving. Regarding 
the CP held, th e hrst portfolio solver (f or solving CSPs only) 
was CPHydra I O’Mahony et al, 2008) that in 2008 won the 
International CSP Solver Compet ition. In 2014, the sequ en- 
tial portfolio solver sunny-cp [ Amadini et al., 2015| at- 
tended the MiniZinc Challenge (MZC) I Stuckey et al, 2m^ 
with respectable results (4* out of 18). To the best of our 


knowledge, no similar tool exist for solving both CSPs and 
COPs. 

In this paper we take a major step forward by introduc¬ 
ing sunny-cp2, a signihcant enhancement of sunny-cp. 
Improvements are manifold. Firstly, sunny-cp2 is paral¬ 
lel: it exploits multicore architectures for (possibly) running 
more constituent solvers simultaneously. Moreover, while 
most of the parallel portfolio solvers are static (i.e., they de¬ 
cide off-line the solvers to run, regardless of the problem 
to be solved), sunny-cp2 is instead dynamic: the solvers 
scheduling is predicted on- line according to a gen eralization 
of the SUNNY algorithm [Amadini et al, 2014c) . Further¬ 
more, for COPs, the parallel execution is also cooperative: 
a running solver can exploit the current best bound found 
by another one through a conhgurable restarting mechanism. 
Finally, sunny-cp2 enriches sunny-cp by incorporating 
new, state-of-the-art solvers and by providing a framework 
which is more usable and configurable for the end user. 

We validated sunny-cp2 on two benchmarks of CSPs 
and COPs (about 5000 instances each). Empirical evidences 
show very promising results: its performance is very close 
(and sometimes even better) to that of the Virtual Best Solver 
(VBS), i.e., the oracle solver which always selects the best 
solver of the portfolio for a given problem. Moreover, for 
COPs, sunny-cp2 using c = 1,2,4,8 cores always out¬ 
performs the Virtual c-Parallel Solver (VPSc), i.e., a static 
portfolio solver that runs in parallel a hxed selection of the 
best c solvers of the portfolio. 

Paper Structure. In Section we generalize the SUNNY 
algorithm for a multicore setting. In Section]^ we describe 
the architecture and the novelties of sunny-cp2, while in 
Section we discuss its empirical evaluation. In Section 
we report the related work before concluding in Section]^ 

2 Generalizing the SUNNY Algorithm 

SUNNY is the algorithm that underpins sunny-cp. Fixed 
a solving timeout T and a portfolio 11, SUNNY exploits 
instances similarity to produce a sequential schedule a = 
[(si, fi),..., (s„, /„)] where solver Si has to be run for sec¬ 
onds and X)r=i ~ input problem p, SUNNY 

uses a k-Nearest Neighbours (fc-NN) algorithm to select from 
a training set of known instances the subset N{p, k) of the 
k instances closer to p. According to the N{p, k) instances, 
SUNNY relies on three heuristics: hgei, for selecting the most 















promising solvers to run; haii, for allocating to each solver a 
certain runtime (the more a solver is promising, the more time 
is allocated); and /isc/u for scheduling the sequential execu¬ 
tion of the solvers according to their presumed speed. These 
heuristics depend on the application domain. For example, 
for CSPs hsei selects the smallest sub-portfolio S' C If that 
solves the most instances in N{p, k), by using the solving 
time for breaking ties, haii allocates to each Si G S a time 
ti proportional to the instances that S can solve in N{p, k), 
while hsch sorts the solvers by increasing solving time in 
N{p, k). For COPs the appro ach is analogous, but d ifferent 
performance metrics are used | Amadini et al, 2014b|. 


As an example, let p be a CSP, If = {si, S 2 , S 3 , S 4 } a port¬ 
folio of 4 solvers, T — 1800 seconds the solving timeout, 
N{p^ k) = {pi, ...,P 4 } the fc = 4 neighbours of p, and the 
runtimes of solver Si on problem pj defined as in Table [TJ 
In this case, the smallest sub-portfolios that solve the most 
instances are {si, S2, S3}, {si, S2, S4}, and {s2, S3, S4}. The 
heuristic hsei selects S = {si,S 2 ,S 4 } because these solvers 
are faster in solving the instances in the neighbourhood. Since 
Si and S4 solve 2 instances and S2 solves 1 instance, haii 
splits the solving time window [0, T] into 5 slots; 2 allo¬ 
cated to Si and S4, and 1 to S2. Finally, hsch sorts the solvers 
by increasing solving time. The final schedule produced by 
SUNNY is therefore ct = [(s 4 , 720), (si, 720), (s 2 , 360)]. 
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T 
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Table 1; Runtimes (in seconds). T means the solver timeout. 


Pcr,c(*) such that: 

i [ ] if i > n 

[(s, T)] if i = ranku{s) A 1 < i < c 

[(s, ^t) \ {s,t) G a A rankcy{s) > c] if i = c 

where Tc = | (s,f) G cr A rank^(s) > c}. 

Let us consider the schedule of the previous example: 
a = [(s 4 , 720), (si, 720), (s 2 , 360)]. With c = 2 cores, the 
parallelisation of a is defined as Po- 2 ( 1 ) = [(s 4 ,1800)] and 
P<t, 2(2) = [(si, 1800/1080 X 720), (sa, 1800/1080 x 360)]. 

The rationale behind the schedule parallelisation P^-.c is 
that SUNNY aims to allocate more time to the most promis¬ 
ing solvers, scheduling them according to their speed. As¬ 
suming a good starting schedule a, its parallelisation P^^c is 
rather straightforward and it is easy to prove that, assuming 
an independent execution of the solvers without synchroniza¬ 
tion and memory contention issues, Va,c can never solve less 
problems than a. 

Obviously, different ways to parallelise the sequential 
schedule are possible. Here we focus on what we think to 
be one of the most simple yet promising ones. An empiri¬ 
cal comparison of other parallelisation methods is outside the 
scope of this paper. 


3 SUNNY-CP 2 

sunny-cp2 solve s both CSPs and COPs encoded in the 
MiniZinc language i Nethercote et al, 2007) , nowadays the 
de-facto standard for modelling CP problems. By default, 
sunny-cp2 uses a portfolio 11 of 12 solvers disparate in 
their nature. In addition to the 8 solvers of sunny-cp 
(viz.. Chuffed, CPX, G12/CBC, G12/FD, G12/LazyFD, 
G12/Gurobi, Gecode, and MinisatID) it includes new, state- 
of-the-art solvers able to win four gold medals in the MZC 
2014, namely; Choco, HaifaCSP, iZplus, and OR-Tools. 


For a detailed de scription and evaluation of SUNNY we 
refer the reader to llAmadini et al, 2014cl |Amadini et al.,\ 
Mir) . Here we focus instead on how we generalized 
SUNNY for a multicore setting where c > 1 cores are 
available and, typically, we have more solvers than cores. 
sunny-cp2 uses a dynamic approach: the schedule is de¬ 
cided on-line according to the instance to be solved. The 
approach used is rather simple: starting from the sequential 
schedule a produced by SUNNY on a given problem, we first 
detect the c — 1 solvers of a having the largest allocated time, 
and we assign a different core to each of them. The other 
solvers of a (if any) are scheduled on the last core, by preserv¬ 
ing their original order in a and by widening their allocated 
times to cover the entire time window [0, T]. 

Formally, given c cores and the sequential schedule a = 
[(si, ti),..., (s„, tn)] computed by SUNNY, we define a to¬ 
tal order such that Si -<a Sj AA ti> tj V {ti = tjAi < j), 
and a ranking function rank^ such that ranka{s) = i if and 
only if s is the i-th solver of a according to Ao-. We define 
the parallelisation of cr on c cores as a function Va,c which 
associates to each core i G {1, ..., c} a sequential schedule 



Figure 1: sunny-cp2 architecture. 

Figure [T] summarizes the execution flow of sunny-cp2. 
It takes as input a problem p to be solved and a list of input 
parameters specified by the user (e.g., the timeout T used by 
SUNNY, the number of cores c to use, etc.). If a parameter is 
not specified, a corresponding default value is used. 
































































The solving process then relies on two sequential steps, 
later detailed: (i) the pre-solving phase, where a static sched¬ 
ule of solvers may be ran and the instance neighbourhood 
is computed; (ii) the solving phase, which runs the dynamic 
schedule on different cores. 

If p is a CSP, the output of sunny-cp2 can be either SAT 
(a solution exists for p), UNS (p has no solutions), or UNK 
(sunny-cp2 can not say anything about p). In addition, ifp 
is a COP, two more answers are possible: OPT (sunny-cp2 
proves the optimality of a solution), or UNB (p is unbounded). 


3.1 Pre-solving 

The pre-solving step consists in the simultaneous execution 
on c cores of two distinct tasks: the execution of a (possibly 
empty) static schedule, and the neighbourhood detection. 

Running for a short time a static schedule has proven to be 
effective. It enables to solve thos e instances that some s olvers 
can solve very quickly (e.g., see I Kadioglu et al, 201 1|) and. 


for COPs, to quickly find good sub-optimal solutions | Ama- 


dini and Stuckey, 20141. 


The static schedule a = ..., (s„,f„)] is a pre¬ 

fixed schedule of n > 0 solvers decided off-line, regard¬ 
less of the input problem p. To execute it, sunny-cp2 
applies a “First-Come, First-Served” policy by following the 
schedule order. Formally, the first m solvers si,... ,Sm with 
m — min(c, n) are launched on different cores. Then, for 
i = m 1,..., n, the solver Si is started as soon as a solver 
Sj (with 1 < j < i) terminates its execution without solv¬ 
ing p. If a solver Si fails prematurely before ti (e.g., due to 
memory overflows or unsupported constraints) then Si is dis¬ 
carded from the portfolio for avoiding to run it again later. If 
instead Si reached its timeout, then it is just suspended: if it 
has to ran again later, it will be resumed instead of restarted 
from scratch. The user has the possibility of setting the static 
schedule as an input parameter. For simplicity, the static 
schedule of sunny-cp2 is empty by default. 

When solving COPs, the timeout defined by the user may 
be overruled by sunny-cp2 that examines the solvers be¬ 
haviour and may decide to delay the interruption of a solver. 
Indeed, a significant novelty of sunny-cp2 is the use of a 
waiting threshold T^. A solver s scheduled for t seconds is 
never interrupted if it has found a new solution in the last 
seconds, regardless of whether the timeout t is expired or not. 
The underlying logic of is to not stop a solver which is ac¬ 
tively producing solutions. Thus, if > 0 a solver may be 
executed for more than its allocated time. Setting to a high 
value might therefore delay, and thus hinder, the execution of 
the other solvers. 

For COPs, sunny-cp2 also enables the bounds commu¬ 
nication between solvers: the sub-optimal solutions found 
by a solver are used t o narrow the search space of the other 
scheduled solvers. In I Amadini and Stuckey, 20141 this tech¬ 
nique is successfully applied in a sequential setting, where 
the best objective bound found by a solver is exploited by the 
next scheduled ones. Things get more complicated in a mul¬ 
ticore setting, where solvers are run simultaneously as black 
boxes and there is no support for communicating bounds to 
another running solver without interrupting its solving pro¬ 
cess. We decided to overcome this problem by using a restart¬ 


ing threshold T^. If a running solver s finds no solution in 
the last Tr seconds, and its current best bound v is obso¬ 
lete w.r.t. the overall best bound v' found by another solver, 
then s is restarted with the new bound v'. The reason behind 
Tr is that restarting an active solver is risky and potentially 
harmful even when its current best bound is actually obsolete. 
However, also ignoring the solutions found by other solvers 
could mean to neglect valuable information. The choice of 
Tr is hence critical: a too small value might cause too many 
restarts, while a big threshold inhibits the bound communi¬ 
cation. Care should be taken also because restarting a solver 
means to lose all the knowledge it gained during the search. 

After performing some empirical investigations, we found 
reasonable to set the default values of T^j and Tr to 2 and 5 
seconds respectively. The user can however configure such 
parameters as she wishes, even by defining a customized set¬ 
ting for each different solver of the portfolio. 

The neighbourhood detection phase begins in parallel with 
the static schedule a execution only after all the solvers of 
a has been started. This phase has lower priority than a 
since its purpose is not quickly finding solutions, but detect¬ 
ing the neighbourhood N(j), k) later on used to compute the 
dynamic schedule. For this reason, the computation of the 
neighbourhood starts as soon as one core is free (i.e., the 
number of running solvers is smaller than c). The first step 
of pre-scheduling is the extraction of the feature vector of p, 
i.e., a collection of numerical attributes (e.g., number of vari¬ 
ables, of constraints, etc.) that characterizes the instance p. 
The feature vector is then used for detecting the set N{p, k) 
of the k nearest neighbours of p within a dataset of known 
instances. 

The default dataset A of sunny-cp2 is the union of a set 
Acsp of 5527 CSPs and a set Acop of 4988 COPs, retrieved 
from the instances of the MiniZinc 1.6 benchmarks, the 
MZCs 2012-2014, and the International CSP Solver Compe¬ 
titions 2008/09. sunny-cp2 provides a default knowledge 
base that associates to every instance pi G A its feature vec¬ 
tor and the runtime information of each solver of the portfolio 
on Pi (e.g., solving time, anytime performance, etc.). In par¬ 
ticular, the feature vectors are used for computing JV(p, k) 
while the runtime information are later used for computing 
the dynamic schedule a by means of SUNNY algorithm. 

The feature extractor used by sunny-cp2 is the same as 
sunny-cp|^ The neighbourhood siz e is set by default to 
k = 70: following i Duda et al., 2000| we chose a value of k 
close to ybr, where n is the number of training samples. Note 
that during the pre-solving phase only N [p, k) is computed. 
This is because SUNNY requires a timeout T for computing 
the schedule a. The total time taken by the pre-solving phase 
(C in Figure [T]) must therefore be subtracted from the initial 
timeout T (by default, T = 1800 seconds as in sunny-cp). 
This can be done only when the pre-solving ends, since C is 
not predictable in advance. 

The pre-solving phase terminates when either: (/) a solver 
of the static schedule a solves p, or (ii) the neighbourhood 


' The feature extractor mzn2feat is available at https : // 
github. com/CP-Unibo/mzn 2fe at For further infor mation 
please see jAmadini et al., 2014a| and jAmadini et al, 2015|. 



















computation and all the solvers of a have finished their ex¬ 
ecution. In the first case sunny-cp2 outputs the solving 
outcome and stops its execution, skipping the solving phase 
since the instance is already solved. 

3.2 Solving 

The solving phase receives in input the time C taken by pre¬ 
solving and the neighbourhood N{p, k). These parameters 
are used by SUNNY to dynamically compute the sequential 
schedule cr with a reduced timeout T — C. Then, a is par¬ 
allelised according to the Pcr,c operator defined in Section]^ 
Once computed Pcr^c. each scheduled solver is run on p ac¬ 
cording to the input parameters. If a solver of Pct,c had al¬ 
ready been run in the static schedule a, then its execution is 
resumed instead of restarted. If p is a COP, the solvers execu¬ 
tion follows the approach explained in Section [UT] according 
to Tyj and Tr thresholds. As soon as a solver of a solves p, 
the execution of sunny-cp2 is interrupted and the solving 
outcome is outputted. 

Note that we decided to make sunny-cp2 an anytime 
solver, i.e., a solver that can run indefinitely until a solution is 
found. For this reason we let the solvers run indefinitely even 
after the timeout T until a solution is found. Moreover, at any 
time we try to have exactly c running solvers. In particular, 
if c is greater or equal than the portfolio size no prediction 
is needed and we simply run a solver per core for indefinite 
time. When a scheduled solver fails during its execution, the 
overall solving process is not affected since sunny-cp2 will 
simply run the next scheduled solver, if any, or another one in 
its portfolio otherwise. 

Apart from the scheduling parallelisation, sunny-cp2 
definitely improves sunny-cp also from the engineering 
point of view. One of the main purposes of sunny-cp2 
is indeed to provide a framework which is easy to use and 
configure by the end user. First of all, sunny-cp2 provides 
utilities and procedures for adding new constituent solvers to 
n and for customizing their settings. Even though the fea¬ 
ture extractor of sunny-cp2 is the same as sunny-cp, the 
user can now define and use her own tool for feature process¬ 
ing. The user can also define her own knowledge base, start¬ 
ing from raw comma-separated value files, by means of suit¬ 
able utility scripts. Furthermore, sunny-cp2 is much more 
parametrisable than sunny-cp. Apart from the aforemen¬ 
tioned Tu, and parameters, sunny-cp2 provides many 
more options allowing, e.g., to set the number c of cores to 
use, limit the memory usage, ignore the search annotations, 
and specify customized options for each constituent solver. 

The source code of sunny-cp2 is entirely written in 
Python and publicly available at https : //github. com/ 
CP-Unibo/sunny-cp 

4 Validation 

We validated the performance of sunny-cp2 by running 
it with its default settings on every instance of Acsp (5527 
CSPs) and Acop (4988 COPs). Following the standard prac¬ 
tices, we used a 10-fold cross-validation; we partitioned each 
dataset in 10 disjoint folds, treating in turn one fold as test set 
and the union of the remaining folds as the training set. 


Fixed a solving timeout T, different metrics can be adopted 
to measure the effectiveness of a solver s on a given prob¬ 
lem p. If p is a CSP we typically evaluate whether, and how 
quickly, s can solve p. For this reason we use two metrics, 
namely proven and time, to measure the solved instances 
and the solving time. Formally, if s solves p in t < T sec¬ 
onds, then proven(s,p) = 1 and time(s,p) = f; other¬ 
wise, proven(s,p) = 0 and time(s,p) = T. A straight¬ 
forward generalization of these two metrics for COPs can be 
obtained by setting proven(s,p) = 1 and time(s,p) = t 
if s proves in f < T seconds the optimality of a solution for 
p, the unsatisfiability of p or its unboundedness. Otherwise, 
proven(s,p) = 0 and time(s,p) = T. However, a major 
drawback of such a generalization is that the solution quality 
is not addressed. To overcome this limitation, we use in ad¬ 
dition the score a nd area metrics introduced in [Amadinil 
and Stuckey, 2014) . 

The score metric measures the quality of the best solu¬ 
tion found by a solver s at the stroke of the timeout T. More 
specifically, score(s,p) is a value in [0.25,0.75] linearly 
proportional to the distance between the best solution that s 
finds and the best known solution for p. An additional re¬ 
ward (score = 1) is given if s is able to prove optimality, 
while a punishment (score = 0) is given if s does not pro¬ 
vide any answer. Differently from score, the area met¬ 
ric evaluates the behaviour of a solver in the whole solving 
time window [0, T] and not only at the time edge T. It con¬ 
siders the area under the curve that associates to each time 
instant ti G [0, T] the best objective value Vi found by s in 
ti seconds. The area value is properly scaled in the range 
[0, T] so that the more a solver is slow in finding good solu¬ 
tions, the more its area is high (in particular, area = T if 
and only if score = 0). The area metric folds in a num¬ 
ber of measures the quality of the best solution found, how 
quickly any solution is found, whether optimality is proven, 
and how quickly good solutions are found. For a fo rmal def¬ 
inition of area an d score we refer the reader to [Amadinil 
and Stuckey, 2014) . 

We ran sunny-cp2 on all the instances of Acsp and 
Acop within a timeout of T = 1800 seconds by varying the 
number c of cores in {1, 2,4,8}. We compared sunny-cp2 
against the very same version of sunny-cp submitted to 
MZC 201^ and the Virtual Best Solver (VBS), i.e., the or¬ 
acle portfolio solver that —for a given problem and perfor¬ 
mance metric— always selects the best solver to run. More¬ 
over, for each c G {1,2,4,8} we consider as additional base¬ 
line the Virtual c-Parallel Solver (VPSc), i.e., a static port¬ 
folio solver that always runs in parallel a fixed selection of 
the best c solvers of the portfolio H. The VPSc is a static 
portfolio since its solvers are selected in advance, regardless 
of the problem p to be solved. Obviously, the portfolio com¬ 
position of VPSc depends on a given evaluation metric. For 
each p G {proven, time, score, area} we therefore de¬ 
fined a corresponding VPSc by selecting the best c solvers 
of n according to the average value of p over the Acsp or 
Acop instances. Note that VPSc is an “ideal” solver, since 


^The source code of sunny-cp is publicly available at https : 
//github.com/CP-Unibo/sunny-cp/tree/mzncl4 













Metric 

sunny-cp 

sunny-cp2 

VPS 

VBS 

1 core 

2 cores 

4 cores 

8 cores 

1 core 

2 cores 

4 cores 

8 cores 

proven (%) 
time (s) 

83.26 

504.10 

95.26 

223.21 

99.11 

136.04 

99.38 

112.60 

99.35 

112.32 

89.04 

297.30 

93.51 

211.13 

95.19 

176.81 

99.24 

68.52 

100 

54.30 


Table 2: Experimental results over CSPs. 


Metric 

sunny-cp 

sunny-cp2 

VPS 

VBS 

1 core 

2 cores 

4 cores 

8 cores 

1 core 

2 cores 

4 cores 

8 cores 

proven (%) 

71.55 

74.40 

74.94 

75.68 

76.34 

61.33 

63.45 

65.60 

75.86 

76.30 

time (s) 

594.79 

501.35 

482.95 

469.74 

454.54 

718.86 

682.35 

645.68 

463.63 

457.00 

score X 100 

90.50 

92.26 

93.00 

93.45 

93.62 

82.80 

85.99 

89.53 

90.79 

93.63 

area (s) 

257.86 

197.44 

149.33 

138.94 

130.53 

314.07 

266.11 

188.20 

178.81 

132.28 


Table 3; Experimental results over COPs. 


it is an upper bound of the real performance achievable by 
actually running in parallel all its solvers. An approach sim¬ 
ilar to VPSc has b een successfully used by the SAT portfolio 
solver ppfolio I Roussel, 2011 1. The VPSi is sometimes 
also called Single Best Solver (SBS) while VPS|n| is exactly 
equivalent to the Virtual Best Solver (VBS). 

In the following, we will indicate with sunny-cp2[c] the 
version of sunny-cp2 exploiting c cores. Empirical re¬ 
sults on CSPs and COPs are summarized in Table |2] and |3] 
reporting the average values of all the aforementioned eval¬ 
uation metrics. On average, sunny-cp2 remarkably out¬ 
performs all its constituent solvers as well as sunny-cp for 
both CSPs and COPs. Concerning CSPs, a major boost is 
given by the introduction of HaifaCSP solver in the portfo¬ 
lio. HaifaCSP solves almost 90% of Acsp problems. How¬ 
ever, as can be seen from the Tables|^and[^ sunny-cp2[i] 
solves more than 95% of the instances and, for c > 1, 
sunny-cp2[c] solves nearly all the problems. This means 
that just few cores are enough to almost reach the VBS 
performance. The best proven performance is achieved 
by sunny-cp 2 [ 4 ]rl Nevertheless, the average time of 
sunny-cp2[8] is shghtly slower. However, difference are 
minimal: sunny-cp 2 [ 4 ] and sunny-cp2[g] are virtually 
equivalent. Note that sunny-cp2[c] is always better than 
VPSc in terms of proven and, for c < 8, also in terms 
of time. Eurthermore, sunny-cp 2 [ 4 ] solves more in¬ 
stances than VPS 4 . In other words, running sequentially 
the solvers of sunny-cp2 on a single core allows to solve 
more instances than running independently the four solvers 
of the portfolio that solve the most number of problems of 
Acsp- Analogously, sunny-cp2[2] solves more instances 
than VPS 4 . This witnesses that in a multicore setting — 
where there are typically more available solvers than cores— 
sunny-cp2 can be very effective. 

Even better results are reached by sunny-cp2 on the 
instances of Acop, where there is not a clear dominant 
solver like HaifaCSP for Acsp. sunny-cp2 outperforms 
sunny-cp in all the considered metrics, and the gain of 
performance in this case is not only due to the introduc- 


^ The practically negligible difference with sunnY-cp2[8] is 
probably due to synchronization and memory contention issues (e.g., 
cache misses) that slow down the system when 8 cores are exploited. 



Eigure 2: Number of COPs for which sunny-cp2 outper¬ 
forms the VBS. 


tion of new solvers. Indeed, the best solver according to 
time and proven is Chuffed, which was already present 
in sunny-cp. Eor each c G {1,2,4, 8} and for each per¬ 
formance metric, sunny-cp2[c] is always better than the 
corresponding VPSc. Moreover, as for CSPs, sunny-cp2 
has very good performances even with few cores. Eor exam¬ 
ple, by considering the proven, time, and score results, 
sunny-cp 2 [ 4 ] is always better than VPS 4 and its score 
even outperforms that of VPSg. This clearly indicates that the 
dynamic selection of solvers, together with the bounds com¬ 
munication and the waiting/restarting policies implemented 
by sunny-cp 2 , makes a remarkable performance differ¬ 
ence. 

Results also show a nice monotonicity: for all the consid¬ 
ered metrics, if c > c' then sunny-cp2[c] is better than 
sunny-cp2[c'] . In particular, it is somehow impressive 
to see that on average sunny-cp2 is able to outperform 
the VBS in terms of time, proven, and area, and that 
for score the performance difference is negligible ( 0 . 01 %). 
This is better shown in Eigure depicting for each perfor¬ 
mance metric the number of times that sunny-cp2 is able 
to overcome the performance of the VBS with c = 1, 2,4, 8 
cores. As c increases, the number of times the VBS is 
beaten gets bigger. In particular, the time and area re¬ 
sults show that sunny-cp2 can be very effective to reduce 
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Chuffed 

CPX 
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Gecode 

OR-Tools 

Other solvers 

sunny-cp2 

^best (s) 

4.05 

52.49 

2.42 

2.87 

3.31 

1.46 

N/A 

5.00 

^best 
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958 

958 

959 

959 

959 

N/A 

958 

time 

T 

T 

T 

T 

T 

T 

T 

10.56 


Table 4: Benefits of sunny-cp2 on a RCPSP instance. T = 1800 indicates the solvers timeout. 


the solving time both for completing the search and, espe¬ 
cially, for quickly finding sub-optimal solutions. For ex¬ 
ample, for 414 problems sunny-cp2[8] has a lower time 
than VBS, while for 977 instances it has a lower area. Ta¬ 
ble reports a clear example of the sunny-cp2 potential 
on an instance of the R esource-Constrained P roject Schedul¬ 
ing Problem (RCPSP) fBrucker et ai, 1999] taken from the 
MiniZinc-1.6 suite Firstly, note that just half of the solvers 
of n can find at least a solution. Second, these solvers find 
their best bound Ubest very quickly (i.e, in a time fbest of be¬ 
tween 1.46 and 4.05 seconds) but none of them is able to 
prove the optimality of the best bound v* = 958 within a 
timeout of T = 1800 seconds. Conversely, sunny-cp2 
finds V* and proves its optimality in less than 11 seconds. 
Exploiting the fact that CPX finds v* in a short time (for CPX 
fbest = 2.42 seconds, while sunny-cp2 takes 5 seconds due 
to the neighbourhood detection and scheduling computation), 
after = 5 seconds any other scheduled solver is restarted 
with the new bound v*. Now, Gecode can prove almost in¬ 
stantaneously that V* is optimal, while without this help it can 
not even find it in half an hour of computation. 


5 Related Work 


The parallelisation of CP solvers does not appear as fruitful 
as for SAT solvers where techniques like clause sharing are 
used. As an example, in the MZC 2014 the possibility of mul¬ 
tiprocessing did not lead to remarkable performance gains, 
despite the availability of eight logical cores (in a case, a par¬ 
allel solver was even significantly worse than its sequential 
version). 

Portfolio solvers have proven their effectiveness in many 
international solvin g competitions. For i nstance, the SAT 
portfolio solvers 3S I Kadioglu et ai, 20111 and CSHC |Mal- 
itsky et ai, 2013} won gold medal s in SAT Competit ion 2011 
and 2013 respectively. SATZilla IXu et ai, 2008) won the 
SAT Challenge 2012, CPHydra ]0’Mahony ef a(., 2008( the 
Constraint Solver Competition 2008, the ASP portfolio solver 
claspfolio I Hops et ai, 2014) was gold medallist in differ¬ 
ent tr acks of the ASP Compet ition 2009 an d 2011, Arvand- 
Herd I Valenzano et ai, 2012) and IBaCoP I Cenamor et al, 
20141 won some tracks in the Planning Competition 2014. 

Surprisingly enough, only a few portfolio solvers are paral¬ 
lel and even fewer are the dynamic ones selecting on-line the 
solvers to run. We are aware of only two dynamic and parallel 
portf olio solvers that attend ed a solving competition, namely 
p3S I Malitsky et al., 2012 1 (in the SAT Challenge 2012) and 
IBaCoP2 I Cenamor et al, 2014| (in the Planning Compe¬ 


tition 2014). Unfortunately, a comparison of sunny-cp2 
with these tools is not possible because their source code is 


"^The model is rcpsp .mzn while the data is in lal 0_x2 . dzn 


not publicly available and they do not deal with COPs. 

Apart from the aforementioned CPHydra and sunn y-cp 
solvers, another portfolio approach for CSPs is Proteus | Hur¬ 


ley et al, 20141. However, with the exception of a prelimi¬ 
nary investigat ion about a CPHydra parallelisation I Yun and 
Epstein, 2012) , all these solvers are sequential and except for 
sunny-cp they solve just CSPs. Hence, to the best of our 
knowledge, sunny-cp2 is today the only parallel and dy¬ 
namic CP portfolio solver able to deal with also optimization 
problems. 

The parallelisation of portfolio solvers is a hot topic which 
is drawing some attention in the community. Eor instance, 
parallel extensions of w ell-known sequentia l port folio ap¬ 
proaches a re studied in i Hoos et al, 2015b) . In iHoos et 
al, 2015^ ASP techniques are used for computing a static 
sched ule of solvers whi ch can even be executed in parallel, 
while I Cire et ai, 2014) considers the problem of parallelis¬ 
ing restarted backtrack search for CSPs. 


6 Conclusions 

In this paper we introduced sunny-cp2, the first parallel CP 
portfolio solver able to dynamically schedule the solvers to 
run and to solve both CSPs and COPs encoded in the MiniZ- 
inc language. It incorporates state-of-the-art solvers, provid¬ 
ing also a usable and configurable framework. 

The performance of sunny-cp2, validated on hetero¬ 
geneous and large benchmarks, is promising. Indeed, 
sunny-cp2 greatly outperforms all its constituent solvers 
and its earlier version s unny-cp. It c an be far better than 
a ppfolio-like approach | Roussel, 201 1] which statically de¬ 


termine a fixed selection of the best solvers to run. Eor CSPs, 
sunny-cp2 almost reaches the performance of the Virtual 
Best Solver, while for COPs sunny-cp2 is even able to out¬ 
perform it. 

We hope that sunny-cp2 can stimulate the adoption 
and the dissemination of CP portfolio solvers. Indeed, 
sunny-cp was the only portfolio entrant of MiniZinc Chal¬ 
lenge 2014. We are interested in submitting sunny-cp2 to 
the 2015 edition in order to compare it with other possibly 
parallel portfolio solvers. 

There are many lines of research that can be explored, both 
from the scientific and engineering perspective. As a future 
work we would like to extend the sunny-cp2 by adding 
new, possibly parallel solvers. Moreover, different paralleli¬ 
sations, distance metrics, and neighbourhood sizes can be 
evaluated. 

Given the variety of parameters provided by sunny-cp2, 
it could be also interesting to exploit Algorithm Config uration 
techniques I Hutter et al, 2011 [ Kadioglu et al, 2010 1 for the 


automatic tuning of the sunny-cp2 parameters, as well as 
the parameters of its constituent solvers. 
















































































Finally, we are also interested in making sunny-cp2 
more usable and portable, e.g., by pre-installing it on virtual 
machines or multi-container technologies. 
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